As is known in the art, many techniques used in the quantitative analysis of objects, such as anatomies and pathologies, from large three-dimensional (3D) volume of imaging data, such as CT data, includes the segmentation of the objects from neighboring objects. One challenge in segmentation comes from the inhomogeneity of the intensity distribution inside the object itself. This makes simple algorithms, like region growing, inapplicable. Edge-based methods also have trouble since there may be strong edges inside various parts of the objects, which becomes local minima. Another major challenge is leakage to neighboring objects. The leakage can be caused by weak edges, and sometimes there are no clear edges even for a human observer. Noise in the data makes this problem even harder since the weak edges will disappear if the data is smoothed to reduce the noise.
In the past decades, the development of level set methods received lots of attention in image segmentation. Many variations of the level set have been proposed, but they can be categorized into region-based and edge-based according to the type of image force or speed used. These methods start with an initial surface and update the surface iteratively towards object boundaries. The segmented surface is smooth due to curvature constraints. Much effort has been spend on the extraction of object surfaces from volume data using level set.
There are several important properties of the level set scheme. First, it automatically handles topological change, although this is of less or no benefit to the segmentation of complex 3D medical structures. Prior knowledge is crucial in the success of such segmentation, and the initial or model or shape should have at least the same topology as the object of interest. In fact, in the surface evolving process from a good initial surface, the algorithm needs to prevent the level set scheme from changing topology, which is an extra effort. The major advantage of the level set method is that it provides a scheme in which the surface can be implicitly represented in discrete volume functions, and can evolve in the format of partial differential equation (PDE) functions. The implementation is simple, and it is provides a flexible platform to design various algorithms. Further, the representation is transparent to dimension, which makes it much more desirable than some other methods that are hard to adapt to 3D cases.
Level set is a frame work in which the designing of speed functions and the detailed implementations leave much space for imagination. Early in the literature, local measures, such as intensity, edge, curvature, and texture have been used. This kind of implementations, however, is still in fact a low level processing method and the problem of local minima and leakage remain unchanged.
It is known that for the above challenges, low level processing alone is not enough and the incorporation of prior knowledge is necessary. In an article by Mitchell, S., C., Bosch, J., G., Lelieveldt, B. P. F., van de Geest, R., J., Reiber, J., H., C., and Sonka, M., entitled “3-D Active Appearance Models: Segmentation of Cardiac MR and Ultrasound Images” published in IEEE Transactions on Medical Imaging, 21(9), (2002), pp. 1167-78, a method based on the active shape model in 3D is proposed for the segmentation,. It is based on point representation of the shape described in an article by Cootes, T., F., Taylor, C., J., Cooper, D., H., and Granham, J., entitled “Active Shape Models—Their Training and Application”, published in Computer Vision and Image Understanding, 61, (1995) pp. 38-59.
Shape is represented by a set of landmark points that are distributed on the object boundaries.
The level set methods provide a good framework to incorporate such prior knowledge since its unified representation of global and local shapes, and do not require point correspondence. Recently, there are works on how to use prior knowledge—particularly global shape priors to augment the level set methods. Among them, Leventon et al. in an article entitled “Statistical shape influence in geodesic active contours” published in Proceedings IEEE Conference on Computer Vision and Pattern Recognition, 1(1), 316-323, proposed a popular methods of Principle Component Analysis (PCA) model of signed distance functions. Simple results are reported for a selected number of different problems.
From our understanding, there are several intrinsic issues associated with the above PCA method when applied to the segmentation of 3D complex structures. The shape prior models use the linear PCA framework, even though the shape model by itself is inherently nonlinear. There has been discussion on why these linear methods work at all on nonlinear problems, as for example described in an article by Tsai, A., Yezzi, A., Wells, W., Tempany, C., Tucker, D., Fan, A, Grimson, W. E., & Willsky, A. (2003), entitled “A shape-based approach to the segmentation of medical imagery using level sets”, published in IEEE Transactions on Medical Imaging, 22(2), 137-154.
It is believed that these methods restrict the search in the shape space to a neighborhood close to the mean shapes. Deviations far away from the mean will lead to strange shapes that are alien to the known object shapes.
The inventor has recognized that for complicated and at the same highly variant 3D shapes, there are several challenges. First, the complexity of the shapes requires many modes; Second, since the shapes are highly variant, large coefficients are needed to fully represent the shape variability, which will drive the algorithm out of the range the linear PCA method is valid. To keep the algorithm in the valid range, there is a preference to restrict the shape in the close neighborhood of the mean shape is restricted and the process relies on the level set method to evolve to the correct surface. This will reduce the power of the PCA method in preventing the local minima and leakage problems.
The first issue, on the other hand, makes training a challenging problem. To cover the complete spectrum of variations, large amount of training samples are needed to generate enough modes. This means the need to manually segment a large set of complex 3D shapes, which is an arduous task by itself. In the implementation level, since there could many combinations of possible modes, the search within the model space becomes an intractable optimization problem. Further, since segmented shape is strictly restricted to the mode space, it is almost impossible to handle shape variations due to pathologies in medical image data. For all of the reasons, using merely shaped based models would be hard to handle 3D segmentation problems effectively and efficiently.
On the other hand, a complex 3D structure usually has strong features. These are the few of high level features that are easily picked up and used as reference for recognition by a human observer. Some of these high level features form prominent and important image features with other structures in their neighborhood. These image features are in fact of global meaning. Whether they are successfully detected will lead to the success or failure of segmentation the complete structure. However, these high level shape features, especially contextual image features are so important that there be an explicit or parametric representation for them and direct the system to look for them, rather than embedding them implicitly in the general shape description.
The high level features differ from what has been used in previous works which use low level features such as local edges or corners found by general detectors. Many of the image features do not reflex the shape features; rather, they are local features within the object. That often provide disturbance rather than help to the segmentation. They are too numerous. The high level features are a few of the image features that belong to the object boundary. They are formed with the 3D context of the object, and need to be detected with prior knowledge. The successful detection of these features is key to the successful application of the level set methods.
Once these high level features are detected, a general method is used to incorporate them into the level set methods.
In accordance with the present invention, a process for anatomical object segmentation, comprises: generation of prior knowledge of the anatomical object to be separated from neighboring objects; and subsequently using of said prior knowledge to level set evolution.
In one embodiment, the prior knowledge includes obtaining a average shape template of the anatomical object.
In one embodiment the prior knowledge includes identifying high level features of the object, such high level features including identifiable interface regions between the object and neighboring objects.
In one embodiment, such high level features also includes identifiable boundary sections of the object surface;
In accordance with the invention, a process is provided for anatomical object segmentation, comprising: generation of prior knowledge of the anatomical object to be separated from neighboring objects; wherein the prior knowledge includes: obtaining an average shape template of the anatomical object and identifying high level features of the object, such high level features including at least one of identifiable interface regions between the object and neighboring objects or boundary sections on the object.
In one embodiment, the process includes subsequently using of said prior knowledge to level set evolution.
In one embodiment, the method includes dividing the template into a plurality of regions and wherein the level set evolution includes using a different image force for each one of the regions.
In one embodiment, the method includes selecting proper image force in each of the sub-region divisions.
In accordance with the present invention, a process includes: generation of knowledge of the anatomical object to be separated from neighboring objects and identification of high level features (i.e., prior knowledge that includes: a mean shape template of the vertebra; and, high level features, i.e., readily identifiable boundary regions between the vertebrae neighboring objects, such as neighboring vertebra and rib structures); and the subsequent use of this prior knowledge to level set.
Prior knowledge of an anatomical object is used for level set segmentation. The prior knowledge is represented with a general shape template and explicit representation of high-level features, such as interface regions between neighboring objects and the object of interest, i.e., the object being segmented from the neighboring objects. The process addresses the local minimum and leakage problem directly. The approach is a general principle that is particularly fit for the segmentation of 3D complex structures. For the sake of clarity, it is introduce in the context of vertebra segmentation from chest CT volume data.
The details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.
Like reference symbols in the various drawings indicate like elements.